On Different Model Selection Criteria In A Forward And Backward Regression Hybrid Network

An assessment of the performance hybrid network with different model selection criteria is considered. These criteria are used in an automatic model selection algorithm for constructing a hybrid network of radial and Perceptron hidden units for regression. A forward step builds the full hybrid network; A model selection criterion is used for controlling the network-size and another criterion is used for choosing the appropriate hidden unit for different regions of input space. This is followed by a conservative pruning step using Likelihood Ratio Test, Bayesian or Minimum Description Length, which leads to robust estimators with low variance. The result is a small architecture that performs well on difficult, realistic, benchmark data-sets of high dimensionality and small training size. Best results are obtained by using the Bayesian approach for the model selection.